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1.0  INTRODUCTION 

Research  being  performed  by  the  Systems  Research 
Laboratory  (SRL)  under  contract  number  N00014-B7-A-0181-0012 
with  the  Office  of  Naval  Research  is  concerned  with  the 
development  of  more  generalized  mathematical  structures  of 
military  processes.  Emphasis  has  been  directed  to  the  modeling 
of  combat  processes  and  the  development  of  associated  allocation 
strategies.  These  efforts  all  assume  perfect  intelligence.  As 
noted  in  the  first  progress  report  (SRL,  1969),  intelligence 
could  reasonably  have  a  large  effect  on  combat  effectiveness 
predictions,  especially  when  one  considers  its  interaction 
with  the  allocation  strategy. 

It  was  thought  that  many  of  the  existing  search  and 
reconnaissance  theories  would  be  useful  for  predicting  the 
amount  of  intelligence-gathering  capability  possessed  by 
a  tactical  unit.  A  thorough  literature  review  in  this  area, 
however,  indicated  that  existing  theories  are  less  than 
useful  for  this  purpose.  Most  of  the  research  efforts  have 
been  devoted  to  the  development  of  strategies  for  the  optimal 
allocation  of  search  effort  and  little  to  the  development  of 
descriptive  models  of  intelligence-gathering  processes.  The 
existing  results  do  not  consider  important  aspects  such  as 
intermittent  target  visibility,  multiple  targets,  moving  targets, 
and  others.  Accordingly,  part  of  the  research  effort  on  this 
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contract  is  being  devoted  to  the  development  of  models  of 
intelligence-gathering  processes . 

The  purpose  of  this  interium  technical  report  is  to 
present  the  results  of  the  literature  review  both  as  a  base 
for  our  research  and  to  indicate  fruitful  areas  of  research 
for  Other  investigators.  Principal  results  in  the  field 
and  the  techniques  used  in  attaining  them  are  presented  in 
an  annotated  bibliography.  A  comprehensive  bibliography, 
organized  under  subject  classifications,  is  included.  Finally, 
some  relevant  areas  for  future  research  are  described. 
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1.1  Defj.nitions  and  Notations 

This  section  contains  some  basic  definitions  and  notations 
used  throughout  the  paper.  Additional  notations  and  exceptions 
to  those  specified  herein  will  be  noted  in  the  text. 

Detection  -  The  act  of  gathering  information  pertaining  to 
th'^  oljcct  being  sought,  the  sifting  out  of 
what  is  important  information  and  the  relaying 
of  that  information  in  some  efficient  form  to 
the  decision  maker. 

Incremental  Detection  Model  -  Let  be  the  instantaneous 
probability  of  detection  on  the  scan  of  an 
area.  Given  n  such  scans,  the  probability  of 
detection  is 

n 

PCD)  s  1  -  7T 
i  =  l 

Continuous  Detection  Model  -  The  probability  of  detecting 

the  target  in  the  interval  (t,  t  +  dt)  is  given 
by  dt.  Given  continuous  observance  over 

an  interval  (0,t),  the  probability  of  detection 
is 

-  /  yCt)  dt 
0 


PCt)  =  1  -  e 


Search  Strategy  -  The  decision  made  on  the  basis  of  information 
obtained  from  the  detection  nnoottoe.  a  "search 
strategy"  will  be  that  set  of  rules  which  asso¬ 
ciates  decisions  with  every  conceivable  result  of 
the  detection  process, 

Target  -  The  object  of  the  search,  a  military  target,  a 

mineral  or  any  other  object  about  which 

information  is  desired. 


Notation : 


prior  probability  of  the  target  being  in  the 
i"*"^  subregion 

the  conditional  probability  of  detection  for 
the  i‘  subregion 


the  conditional  overlook  probability  for  the 

4"  Vi 

i^  subregion 

a.“0 

a  search  strategy  (possibly  infinite)  where  Oj 
denotes  which  region  is  to  be  searched  on  the 
j^^  trial 

+*v 

cost  of  searching  the  i^  subregion 


the  time  spent  searching  the  i^  subregion 
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1.2  Classification  of  Detectors  and  Targets 

Models  of  search  and  reconnaissance  processes  treat 
detectors  and  targets  with  varied  combinations  of  properties 
or  assumptions  regarding  their  behavior.  This  section 
presents  a  classification  of  analytic  assumptions  that  may 
be  used  to  describe  the  behavior  of  detectors  and  targets. 

Deteotora 

1.  Single  Detector  with  a  Single  Scan 

a)  Binary  detection  (Incremental  Detection  Model) 

b)  Interval  detection  (Continuous  Detection  Model) 

1)  Non-cumulative  probability  of  detection 

2)  Cumulative  probability  of  detection 

(a)  partial  loss  of  information 

(b)  no  loss  of  information 

2.  Single  Detector  with  Multiple  Scan  Capability 

a)  Binary  detection 

b)  Interval  detection 

1)  Non-cumulative  probability  of  detection 

2)  Cumulative  probability  of  detection 

(a)  partial  loss  of  information 

(b)  no  loss  of  information 
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3.  Multiple  Detectors  with  Single  Scan  Capability 

a)  Binary  detection 

1)  Detectors  act  independently 

2)  Detectors  act  dependently 

b)  Interval  detection 

1)  Independent  action 

(a)  non-cumulative  probability  of  detection 

(b)  cumulative  probability  of  detection 

(1)  partial  loss  of  information 

(2)  no  loss  of  information 

2)  Dependent  action 

(a)  non-cumulative  probability  of  detection 

(b)  cumulative  probability  of  detection 

(1)  partial  loss  of  information 

(2)  no  loss  of  information 

4.  Multiple  Detectors  with  Multiple  Scan  Capability 

a)  Binary  detection 

1)  Independent  action 

2)  Dependent  action 

b)  Interval  detection 

1)  Independent  action 


(a)  non-cumulative  probability  of  detection 

(b)  cumulative  probability  of  detection 
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(1)  partial  loss  of  information 

(2)  no  loss  of  Information 
(2)  Dependent  action 

(a)  non-cumulative  probability  of  detection 

(b)  cumulative  probability  of  detection 

1}  partial  loss  of  informaticn 
2)  no  loss  of  information 

Targets 

1 .  Single  Target  (which  may  be  an  entire  group) 

a)  The  target  can  exhibit  binary  visibility,  i.e., 
it  is  either  visible  or  not  with  specified 
probability . 

b)  The  target  may  have  only  a  single  interval  of 
visibility,  the  length  of  this  interval  having 
a  known  probability  density  function. 

(1)  The  single  visibility  interval  can  begin 
at  time  t  =  0. 

(2)  The  single  visibility  interval  can  begin 
at  some  time  t  ^  0. 

c)  The  target  can  exhibit  multiple  periods  of 
visibility . 


e 


SEARCH  PROCESS 


DETECTOR 
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noted  below.  Categories  C,  D,  and  E  are  discussed  in 


PVa  O  0  mv^iA  U  ^  "L  •  5_V 


■  M  W  W*A  V 


research  are  described  in  Chapter  5. 


(A)  General  Discussion 

This  category  includes  the  pioneering  work  of 
Koopman,  the  applications  of  Morse  and  Kimball, 
and  the  bibliographies  of  Dobbie  and  Enslow. 


(B)  Measures  of  Performance 

This  category  includes  papers  which  consider 
various  search  objectives.  Although  it  is 
usually  assumed  that  the  objective  of  search 
is  the  detection  of  the  target,  other  objectives 
such  as  maximizing  the  information  gain  have  been 
proposed  and  studied. 

(C)  Allocation  of  Effort  in  One-Sided  Search 

This  category  considers  the  problem  of  the  distri¬ 
bution  of  effort  required  to  find  a  target  when 
the  distribution  of  the  target  is  known  to  the 
searcher.  These  subheadings  are  included  under 
this  category. 

( 1 )  Stationary  Targets 

The  target  is  assumed  stationai''y  although 
some  authors  consider  targets  that  suddenly 
appear  and  remain  visible. 


The  size  and  ahan*  of  the  target  may  have 
some  effect  cn  the  formulation  and  solution 
of  the  problem. 


(3)  Moving  Targets 

The  target  is  moving  without  conscious  evasion 
and  the  searcher  knows  the  motion  or  distribution 


of  motion. 


<D)  Two-Sided  Search 

This  category, which  considers  the  game  theoretio'aspects 
of  search,  investigates  the  search  problem  with  a 
conscious  evader.  Included  in  this  category  are  the 
search/evasion  problems  in  which  the  searcher  and 
evader  can  alter  their  motions  differentially  by 
choices  of  continuously  varying  parameters,  e.g., 
the  theory  of  differential  games  as  formulated  by 
Isaacs. 


(E)  Miscellaneous 

This  category  includes  papers  containing  important 
results  in  the  development  of  search  theory  and 
methodology  or  application  of  search  concepts  to 
the  operations  of  reconnaissance  and  surveillance. 
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2.0  ALLOCATION  OF  EFFORT  IN  ONE-SIDED  SEARCH 

The  purpose  of  this  chapter  is  to  summarize  the  pub¬ 
lished  results  obtained  to  date  in  regard  to  the  allocation 
of  effort  in  the  one-sided  search  for  a  stationary  target. 

In  this  context, the  distribution  of  the  target  is  known  to 
the  searcher  although  it  may  not  be  present  at  the  start  of 
the  search.  The  major  results  of  investigations  in  this  area 
are  presented,  as  well  as  the  interrelationships  between 
them,  if  any.  Since  the  entire  field  seems  to  have  originated 
from  the  investigations  of  B.  Koopman,  these  results  will  be 
the  starting  point  of  the  review. 

Koopman  (1946)  describes  two  types  of  detection  processes} 
the  "glimpse"  or  discrete  mode,  and  the  continuous  mode.  In  the 
former,  one  has  a  single  scan  or  glimpse  probability  of  detection 
which  may  be  functionally  dependent  upon  range,  time,  etc. 
Given  n  such  looks,  the  probability  of  detection  is  determined  as 

n 

p„  =  1  -  Tf  . 

i=l 

The  continuous  mode  is  characterized  by  the  assumption  that  the 
probability  of  detection  in  a  short  time  interval  of  length  dt 
is  given  by  Y(t)dt.  Given  continuous  searching  over  a  time 
interval  of  length  t,  the  probability  of  detection  is  given  by 

Y(t)dt 


r(t)  =  1 


e 


The  optimal  allocation  of  searching  effort  for  a  stationary 
tarflat  was  derivad  by  Knonman  i  n  the  following  '  fsShicr.  t 
Let  a  stationary  target  be  located  in  a  known  region  A  with 
known  probability  density  function  p(k,y)  continuous  in  the 
region  A  with  the  properties 

Min  (x,y)  =  p  >0, 

A  o 


p(x,y)dxdy 


1. 


Assume  the  searcher  has  certain  constraints  on  the  amount  of 
effort,  that  can  be  allocated  to  the  search.  Consider  a 
search  aensity  function  -JiCxjy)  defined  on  the  region  A  with 
the  properties  that 


i(>(x,y)dxdy  =  # 


A 


(j)(x,y >  >  0  on  A. 


(1) 

(2) 


Assume  further  that  the  searcher  is  operating  in  the  continuous 
detection  mode.  Then  the  probability  of  detecting  the  target, 


p[(fi],  is  given  by 


pC<t)]  = 


ff  p(x,y)  ^1  - 


-ij)(x, 
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The  conditional  probability  of  detection,  1  - 
is  the  result  of  the  two-dimensional  "law  of  random  search." 
The  fundamental  ^i-ublem  is  to  determine  from  among  all  the 
functions  satisfying  equations  1  and  7  that  which  gives  3 
its  maximum  value. 

Koopman  obtains  the  optimal  solution  as 

(|)<x,y)  =  log  p(x,y)  -  i  fh  p(x,y)dxdy  +  ♦/A, 

A 


for 


and 


where 


<x,y)e  A, 


(|)(x,y)  =  0  for  (x,y)c  A-A^ 


A  -  |(Xiy)  |p(x,y )  ^  b  »  log  b  -  i  Jf  log  p(x,y)  dxdy  t  -J  =  o) 

'  ^  K  A  I 

By  considering  A  =  A^  +  Aj  +  • • "A^,  one  can  obtain  the  solution 
to  the  n  region  search  problem.  Some  generalizations  suggested 
by  Koopmanj  include  the  case  of  visibility  varying  from 
position  to  position,  the  case  of  weighting  the  probability  of 
detection  by  a  function  dependent  upon  where  the  target  is 
detected,  and  weighting  the  search  density  function  by  a  cost 
function  dependent  upon  the  region  bt^ng  searched. 

Charnes  and  Cooper  ^1958),  develop  an  algorithm  for  the 
solution  of  a  discrete  version  of  Koopman 's  problem. 
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Let  {Pj}»  j  *  1»  2,  •••,  N  denote  the  probability  that  the 
target  is  in  the  j region.  Then  if{(|>j},  j  =  1,  2,  •••,  N 
denotes  the  normalized  search  density  vector,  Koopman's 
problem  becomes 

Min  ‘ 

j  =  l 

N 

S.T.  =1,  (f.  >  0, 

j  =  l 

N 

and  Pj  ^  0,  Pj  =  1, 

where  $  is  a  scale  factor  relating  the  allocations,  ♦j ,  of 
search  effort  to  the  total  amount  of  search  effort  available. 

The  algorithm  is  obtained  from  the  application  of  the  Kuhn- 
Tucker  conditions  for  optimality  to  the  above  convex  programming 
problem. 

The  detection  processes  in  Koopman's  formulations 
were  quite  restrictive,  de  Guenin  (1961)  generalized  these 
processes  as  follows  :  Let  p[<^<x)]  denote  the  probability  of 
detecting  the  target  with  an  effort  'J*(x)  when  the  target  is 
at  X.  The  following  assumptions  are  made  with  r'='spect  to 
pr<Ji(x)  ] , 

(1)  p(o)  =  0 

(2)  p'  (it)  =  >0 

(3)  p'Ct)  =  a  decreasing  function  of  <i> 

(4)  p'(o)  2l  0,  p'(*)  =  0, 
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From  the  above  properties  p '(<>>)  admits  of  an  inverse  function 
v  =  fCp‘)t  The  basic  problem  becomes 


Max  P 


g(x)p[(<»(x)  ]dx 


S.T.  (Kx)  ^  0 

())(x)  = 

-.00 

where  g(x)  is  the  probability  density  functioxi  for  target 
location.  de  Guenin  derives  the  following  necessary 
conditions  for  optimality  under  the  above  assumptions* 

Theorem  !  A  necessary  condition  for  p  to  be  optimum  is  that 
at  any  point  x  such  that  ‘f'Cx)  >  0, 


g(x)  p'WCx)]  =  constant 

s' 

gCx)/^  (fi/dp^  =  constant, 

where  d  <l>/dp  =  the  marginal  effort  to  increase  the  detection 
probability.  One  might  restate  this  result  as  follows  : 
Whenever  the  distribution  of  effort  is  optimum,  the 
marginal  effort  required  to  increase  the  detection 
probability  at  any  point  is  proportional  to  the  proba¬ 
bility  density,  iCx),  of  the  location  of  the  object. 
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Koopman  (1946)  observed  that  the  distribution  which 
maximizes  the  detection  probability  with  a  given  amount  of 
effort  has  the  interesting  property  that  it  is  the  sum  of 
conditionally  optimal  distributions.  That  is,  the  optimal 
distribution  of  +  E2  is  the  sum  of  the  optimal  distribu¬ 
tion  of  and  the  conditionally  optimal  distribution  of  E,, 
given  that  the  target  has  not  been  found  with  the  previous 
distribution  of  E^.  Bobbie  (1963)  develops  sufficient  con¬ 
ditions  for  this  additive  property  to  hold,  then  shows  that 
the  solution  to  this  class  of  problems  can  be  attained  by 
"optimizing  conditionally  in  the  small."  Let  p(x,f(x)),  be 
the  condtional  probability  that  a  target  at  x  will  be  detected 
by  the  searching  effort  of  intensity  f(x)  at  x.  If  the  de¬ 
tection  rate,  k(x),  is  independent  of  the  searching  effort 
f(x),  then  Dobbie  shows  that 

p(x,f(x))  =  1  -  expC-k (x)f (x) ] . 

Furthermore,  it  is  also  shown  that  if  ^  is  s  positive 

monotonic  non- increasing  function  of  f  for  every  x,  then  the 
distribution  obtained  by  maximizing  the  probability  of  detection 
in-the-small  will  maximize  the  overall  detection  probability. 

It  is  also  shown  that  the  expected  effort  required  to  detect 
the  target  is  given  by 


r  = 


C)(E)dE 


(1-p (E) )dE, 
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p(E)  is  the  probability  of  detecting  the  target  with  effort 
‘b'ited  according  to  “  particulax'  uiatribvitiop  runction. 
From  the  above  equation,  one  can  see  that  the  expected  effort 
is  minimized  by  always  distributing  the  effort  to  maximize 
the  probability  of  detection  with  the  effort  expanded  tlms 
far.  In  contrast,  the  distribution  that  maximizes  the  proba¬ 
bility  of  detection  with  a  given  amount  of  effort  can  be 
non-optimal  for  all  values  of  effort  less  than  the  total,  as 
long  as  the  schedule  attains  the  final  distribution  when  all 
the  effort  has  been  applied. 

Pollock  (1960)  introduces  a  discrete  search  model  for 
two  regions  and  determines  the  optimal  sequential  strategies 
for  this  model.  A  single  searcher  is  given  the  a  priori 
probability  P  that  the  target  is  in  region  1.  Conditional 
detection  probabilities  and  q^  are  also  given.  It  is 
assumed  that  each  glimpse  in  either  region  takes  one  unit  of 
time.  As  the  search  progresses,  the  a  posteriori  probabilities 
are  obtained  using  Bayes’  theorem.  For  example,  suppose  the 
searcher  is  unsuccessful  in  his  look  into  region  1,  the  a 
posteriori  probability  that  the  target  is  in  that  region 
is  given  by 

(1-q, )p 

p.  =  - ^  , 


Similarly,  the  A  posteriori  probability  of  the  target  being  in 
region  1,  given  an  unsuccessful  look  into  region  2,  is 


The  optimal  value  of  E(p)  under  the  above  strategy  is  determined 
via  a  "bootstrap"  technique  of  extending  the  region  in  which  the 
optimal  value  of  E(p)  is  known# 
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In  comparing  the  optimal  values  of  the  expected  length 
of  search  between  the  discrete  and  continuous  models,  Pollock 
observed  that  for  small  values  of  q  they  are- very  close 
indeed;  however,  as  q-»-l  they  become  quite  different.  He  also 
observed  that  the  criteria  of  (1)  maximizing  the  probability 
of  detection  by  the  end  of  a  fixed  time;  and  (2)  minimizing 
the  expected  length  of  time  until  detection;  lead  to  the  same 
results  for  the  allocation  of  effort. 

Gilbert  (1959)  considers  the  continuous  version  of  the 
two-box  search  problem  including  non-zero  switching  times. 

First,  he  notes  that  in  generals  search ,  under  the  assumptions 
made  up  to  this  point  in  our  discussion,  may  be  compared  to 
a  one-person  game.  Although  not  solvable  as  such,  he  concludes 
that  all  attention  should  be  restricted  'to  pure  (deterministic) 
strategies,  rather  than  mixed  (probabilistic)  strategies.  He 
also  notes  that  with  p  =  1  -  p  =  1/2,  and  q^^Ct)  *  =  1  -  e“^, 

it  is  optimal  to  switch  from  one  box  to  another.  wheheV'er  'the  box 
being  searched  has  received  a  longer  time  of  sea '•oh  f  ,  the 
other  box.  Then,  by  switching  from  box  to  box  ra,|f'iaiv:  enough, 
one  can  get  expected  search  times  as  close  to  two  aa  desired. 

This  leads  to  the  definition  of  a  "limit  strategy,"  which  will 
approach  the  true  optimum  strategy  in  the  limit  as  switching 
becomes  instantaneous.  The  limit  strategy  is  defined  as  a 
pair  of  monotone  no.n-decreasing  functions  x(t)  and  y(t)  such 


that 
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(b) 


For  (l-p)a2  _<  pa^^,  first  look  in  box  1  for 


i-  In  1 1  -  r  units  of  time,  then  follow 
OjLl-F  J 

the  limit  strategy. 

Allowing  for  a  non-zero  switching  time  S,  the  strategies  of 
interest  become  those  which  follow  “staircase"  paths  in  the 
<x,y)  plane.  If  switches  occur  at  the  points  (x^jy^),  i  =  li 
2,  then  the  expected  search  time  of  a  strategy  is 

E  =  J  Q(x,y)(dx  +  dy)  +  S 

i=l 

where  the  integral  is  a  line  integral  taken  along  the  staircase 
path.  Gilbert  develops  the  following  theorems  pertaining 
to  this  case. 

Theorem  Let  C  be  a  line  segment  between  two  switch 

points  (x^,  and  of  a  mini¬ 

mizing  strategy.  If  C  is  horizontal,  there 
must  be  points  on  C  at  which  Qv  -  Q,,  — 

If  C  is  vertical,  there  exist  points  of  C 
at  which  Qv  “  Qv  — 

Tneorem  Let  p  =  1-p  =  1/2,  ~ 

let  the  distribution  function  1  -  q(t)  have 
mean  T.  Then  bounds  on  the  minimum  expected 
time  Eq  are  given  by 


T  +  s/2  <  Eq  <  2T  +  s/2  +  y  2ST 

The  above  theorems  will  yield  the  optimal  solutions  in 
special  cases.  For  the  case  in  which  =  l-e“^,  and 
Qj  =  l-e”^,  Gilbert  determines  that  the  switch  points 
are  (w,o),  (w,  2w),  (3w,  2w)  where  w  satisfies  the  equation 

S  +  =  sinh  w„, 

0  o 

and  the  minimum  value  of  E  is 

E„  =  1  +  cosh^(w^/2). 

o  o 

However,  these  results  will  not  yield  solutions  in  more 
general  situations.  Kisi  (1966)  obtained  the  same  result 
independently  using  somewhat  more  direct  arguments. 

Blachman  (1959)  considers  the  following  variation  of  the 
search  problem  formulated  by  Koopman.  The  object  is  not  present 
at  the  beginning  of  the  search  but  has  a  distribution  of 
arrival  times,  and  the  aim  is  not  to  maximize  the  probability 
of  detection  but  to  minimize  the  expected  delay  between  arrival 
and  detection.  An  object  may  appear  in  any  one  of  n  locations 
and  will  thereafter  remain  there,  the  probability  of  the  i"*"^ 
location  being  pj^,  with  ^  p£  =  1.  The  time  of  appearance  of 
the  object  is  distributed  uniformly  over  a  long  interval  of 
length  T.  A  look  in  the  i^  location  takes  a  time  t^  and,  if 
the  object  is  there,  the  look  detects  it  with  probability  q^. 


The  basic  question  is:  In  what  order  should  the  various 
locations  be  scanned  during  the  time  T  to  minimize  the 
expected  delay  between  the  appearance  of  the  object  and  its 
detection?  The  search  pattern  is  characterized  by  the  inter¬ 
vals  T^j  between  the  beginning  of  the  (j-1)®^  look  in  the 

i  location  and  the  beginning  of  the  j  .  T^^  is  defined  as 

the  interval  between  the  start  of  the  search  and  the  first 

look  in  the  i"^^  location.  It  is  assumed  that  the  target  will 
not  appear  before  the  start  of  the  search. 

For  a  given  search  procedure,  the  expected  delay  between 
the  arrival  and  the  discovery  of  the  object  is 


'■5'‘ 


T.  .  + 


Where  is  the  total  number  of  looks  in  the  i^^  location  and 
=  1-q^  is  the  probability  of  failing  to  detect  the  target. 

The  procedure  is  to  choose  positive  quantities  T^j  that  minimize 
the  expected  delay  subject  to 


^T  (i=l,  2,  •••,n). 


If 
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Minimizing  the  expected  delay  subject  to  the  above  constraint 
yields,  treating  as  fixed, 


(l-ai)Ji 


i  *  1,  J. 


under  the  assumption  that.  T  is  great  enough  so  that  a]i 
are  large.  The  optimum  expected  delay  is  given  by 


(5) 


To  determine  the  optimum  J^,  (5)  must  be  minimized  subject 
to 


N 


E  ''i 

i  =  l 


T. 


The  results  of  this  minimization  are: 


(a) 


t  . 

mxn 


In  general,  it  is  not  possible  to  arrange  a  search  pattern 
that  satisfies  the  above  conditions  for  all  locations, because 
the  condition  that  ''looks"  in  different  locations  must  not 
overlap, has  not  been  taken  into  account.  Hence,  one  can 
conclude  only  that  a  search  pattern  which  approximately 
satisfies  the  above  condition  is,  at  least,  approximately 
optimum. 

Blachman  and  Proschan  (1959)  consider  the 'r'  l lowing 
general  search  problem.  Objects  arrive  in  accordance  with 
a  Poisson  process,  the  rate  of  arrival  being  A.  Having 
arrived,  an  object  appears  (and  remains  until  detected)  in 
box  i  with  probability  p^.  A  single  scan  of  box  i  costs 
c^  (possibly  including  the  cost  of  false  alarms),  takes  time 
tp  and,  if  the  object  is  presenx  in  box  i  at  the  beginning 
of  the  scan,  will  detect  it  with  probability  q^.  The  resultant 
gain,  gji(t),  i  =  1,  2,  •••,  n,  is  a  non-increasing  function  of, 
t,  the  delay  between  arrival  and  the  beginning  of  the  detecting 
look.  Considering  only  cyclic  search  scheduler,  i.e.,  search 
schedules  which  repeat  after  D  units  of  time,  where  D  is  arbi¬ 
trary,  the  authors  derive  the  optimum  search  procedure.  The 
expected  net  gain  per  unit  of  time  from  f^  regularly  spaced 
looks  per  unit  of  time  allocated  to  box  i,  i  =  1,  2,  •••,  n, 
is  given  by 


(m+l)7f^ 


N  r  ”  m  f  n 

S  S  •  ■’i’  /  -  ‘'i=i 

isl  L"**°  I 

ra/f.  J 


-E 

i=l 


The  problem  is  to  maximize  the  expected  net  gain  subject  to 


N 

<1,  >  0,  (i  =  1,  2.  n). 

i=l 


The  solution  to  the  above  problem  is  obtained  as  follows: 
Define.  f^(r)  fpr  r  •>  0  as 


f£<r) 

=  0 

if 

r^<0) 

<  r 

fi(r); 

',.S  90 

if 

/  . 

^  r  . 

f.  <r) 

=  ^i 

such 

that 

ri(fi)  = 

=  t:;; 


where 
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Also  define 

N 

F(r)  = 

i»l 

r*  =  min  r, 
r  >  0 
F<r)  <  1 

and 

=  f(r*). 

The  major  result  of  this  paper  is  the  following  theorem; 
Theorem: 

<i  =  1,  2,  •**,  n)  maximizes  r  among 

f^  I  i  =  1,  2,  *•',  n  satifying 

N 

E 

i=l 

In  the  foregoing  discussion,  the  optimal  schedule  was  obtained 
by  ignoring  conflicts  among  boxes,  however,  the  authors  show 
that  by  taking  n  sufficiently  large,  and  at  the  same  time  the 
Pi’  "*^1’  ‘^i  small  one  can  always  produce  a 

conflict-free  schedule  with  the  expected  gain  per  unit  of 
time  as  close  as  desired  to 


r*  =  rcf^S  •••,  f^*). 

Some  additional  results  obtained  in  this  paper  include: 

(a)  In  considering  how  best  to  schedule  scans  in  any 
one  given  box  (ignoring  all  other  boxes,  for  the 
moment),  the  optimum  schedule  calls  for  scans 
uniformly  spaced  in  time. 

(b)  The  following  theorem  provides  a  sufficient  condi¬ 
tion  for  answering  the  question:  Under  what  oiroum- 
stances  should  a  given  box  be  searched? 

Theorem; 

00 

If  J"  tdg^(t)  s  -  w,  then.f^*  >  0, 
o 

(o)  By  taking  the  gaii';  function  to  be  the  negative  of 
the  delay  between  arrival  and  detection,  i.e., 

g^Ct)  =  -  (t,+  t^.),  i  =  1,  2,  ••’,  n, 

the  optimal  frequency,  is  shown  to  be  proportional 


to 


Matula  (1964)  derives  conditions  for  the  existence  of  an 

ultimately  periodic  search  program  in  the  following  context; 

An  object  is  in  one  of  a  finite  set  I  of  possible  locations, 

with  a  priori  probability  p.,  2^  P,*  =  1*  Associated  with 

^  i=I  ^ 

each  location  i  is  a  cost  for  searching  that  location,  c^, 
and  an  overlook  probability,  a^,  if  the  object  is  in  i  and 
i  is  searched,  it  is  not  detected.  The  problem  is  to  find 
a  program  cf  =  (o(l),  c(2),  •••),  i.e.,  a  sequence  of  locations 
lo  be  searched  such  that  the  expected  cost,  v(o)  of  finding 
the  object  is  minimal.  A  program  is  called  ultimately  periodic 
if  a(j  +  6)  =  a(j)  for  all  j  >  T,  where  T  denotes  the  length 
of  the  transient  phase  and  0  the  length  of  the  period. 

The  major  result  of  this  paper  is  the  conditions  for 
the  existence  of  an  ultimately  periodic  optimal  program  as 
well  as  the  minimal  period  and  the  minimal  transient  length. 

It  is  to  be  noted  that  the  general  dynamic  programming  solution 
gives  an  optimal  program  recursively,  whereas  the  results  of 
this  paper  have  the  advantage  of  yielding  a  closed  form  expression 
and  require  evaluation  of  only  the  first  T  +  9  terms.  In  addi¬ 
tion,  a  periodic  optimal  program  yields  for  the  expected  cost 
a  power  series  that  is  algebraically  summable  in  closed  form. 

The  results  of  the  paper  are  summarized  in  the  following 
assertions : 

Lemma  If  a  is  an  ultimately  periodic  optimal  program  of 

length  T  and  period  0  =2  where  is  the 

i=T 


transient 
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(b)  o^is  optimal 

(c)  0  is  the  minimal  possible  period 

(d)  T  is  the  minimal  transient  length. 

Combining  the  previous  results ,  Matula  obtains : 

Periodic  Search  Theorem;  ^ 

A  necessary  and  sufficient  condition  for  the  existence 
of  an  ultimately  periodic  optimal  program  is  that  the  ratios 

jlog  a^y/log  dj  I  (i,j)  E  I, 
all  be  rational. 

It  is  interesting  to  note  that  the  limiting  frequency  of 
search  of  a  location  for  any  optimal  program  depends  only 
upon  the  overlook  probabilities,  not  on  the  initial  probability 
distribution  or  even  the  relative  costs. 

In  the  following  example,  due  to  Klein  (1968),  one  can 
note  the  more  detailed  structure  of  the  transition  mechanism. 
Klein  considers  the  following  problem.  An  object  moves  about 
within  a  finite  number  of  regions,  one  per  time  unit,  according 
to  known  probabilistic  laws.  A  single  searcher,  using  a  de¬ 
tection  system  whose  effectiveness  is  a  function  of  the  amount  of 
effort  used  and  the  region  searched,  checks  one  region  at  a 
time  until  the  object  is  found,  his  budget  effort  is  exhausted, 
or  he  decides  that  it  is  "uneconomical"  to  continue.  The 
problem  is  to  find  an  optimal  sequential  search  policy,  i.e.. 
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one  which  tells  the  searcher,  at  each  point  in  time,  whether 
to  search,  where  to  search,  and  how  much  effort  to  use.  It  is 
further  assumed  that  the  target's  movements  are  independent  of 
its  location  and  that  the  searcher  is  "noisy"  enabling  the 
target  to  base  his  movements  on  knowledge  of  the  searcher's 
location  at  the  end  of  each  period.  The  following  assumptions 
are  also  made: 

(a)  L  +  1  regions  are  to  be  searched,  0,  1,  •**,  L. 

(b)  The  searcher  starts  in  region  0  (the  base)  and  the 
object  is  in  any  region.  The  budget,  of  size  B, 
consists  of  a  finite  number  of  discrete  units. 


The  two  classes  of  states  and  their  associated  labels  are 
defined  as: 

(1)  ijj^,  region  i  has  been  searched,  0  =4 'J^^successfully , 
and  b  units  of  the  budget  remain  for  further  use. 

(2)  same  as  above,  except  1  =4  successful  search. 

The  state  space  of  the  decision  process  is  given  by 


j  ^  = 


0,  •••,L;b=0,  1,  ••• 


,B-1;  a=0,l|  U  |og  | 


where  Og  is  the  initial  state. 


It  is  assumed  that  the  target  discovers  the  searcher's 


location  at  the  end  of  each  period.  His  evasion  strategy, 
based  on  this  information,  is  assumed  to  be  randomized  and 
represented  in  the  form  of  a  stochastic  matrix 
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where  i  denotes  the  searcher's  current  location  and  j  the 
target’s  next.  Then,  corresponding  to  each  searcher  position 
(i),  the  target  moves  to  position  j  with  probability  h^^ . 

Note  that  this  implies  that  the  target's  ability  to  move  is 
independent  of  its  location.  This  may  not  be  true  of  the 
searcher's  mobility.  The  effectiveness  of  the  searcher's 
detection  process  depends  upon  the  region  searched  and 
the  amount  of  effort  used,  i.e., 

Vj(e)  =  probability  that  a  search  of  region  j 

using  effort  e  will  find  a  target  if  it 
is  in  the  region. 

After  each  determination  of  the  current  state  of  the 
decision  process,  say  the  searcher  chooses  a  decision, 
jg,  from  a  finite  set  k(i^),  i.e.,  the  searcher  chooses  the 
next  region  to  be  examined  (j )  and  the  amount  of  effort  to 
be  used  (e  =  1 ,  2 ,  • • • ,  b) .  It  is  assumed  that  the  decision 
is  made  with  probability  d(i^,jg).  The  process  is  controlled 
by  a  randomized  stationary  decision  rule  (Derman  (1962)  has 
shown  that  attention  may  be  restricted  to  this  class  of  rules): 

=  I  d(i^,  j^)  I  ,  d(i“,  j^)  >  0. 

£  d  (ib.jg)  =  1- 


D 
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The  Droblem  Is  tn  s<*i 


» e»  ^  «A«  •  1  ^  r\  ^  —  j  m 
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of  all  randomized  stationary  rules.  Next,  the  stopping  states 
for  the  chain  are  defined  as  follows: 

(a)  let  A  equal  the  set  of  all  states  in  which 
the  target  is  found 


hii  t  .  0.  •••.L; 


;b=o,  1,  -..jB-l 


<b)  let  G  equal  the  set  of  all  states  in  which  the  budget 


is  exhausted, 


'  ^  i'o  «  i  •••.  L  I  , 

then  T  =  AUG  is  the  complete  set  of  stopping  states  for  the 
chain.  It  is  also  assumed  that  the  process  starts  in  state 

O 

Og,  with  probability  1.  The  transition  probabilities  for 
the  controlled  chain,  p(i“,  j°)  follow; 

Let  r^j  =  1,  2,  be  the  travel  effort  needed  to  go 

from  i  to  j ;  then,  for  all  integers  b,f:  0  <  f  =  b-e-r^^  <  b  <  B, 

=  ^ij  v.(e)d(i“,  j^),  i®  e  s„T,  e  A, 


0  0 


1  -  h^.  v.(e) 


h  ^  S-T ,  J  -  e  S' 
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where  from  Markov  chain  theory  l/ir(Og)  is  the  mean  recurrence 
time  for  state  Og  and 

E(t(D))  =  — iw-  -  1 

.(0g> 

is  expected  duration  of  the  search.  A  successful  search  termif 
nates  in  class  A,  hence  the  probability  of  a  successful  search 
using  rule  D  is 


Klein  offers  the  following  objective  functions  for  consideration : 

(a)  min  Q(D) 

{D} 

S.T.  P<D)  >  e 

(b)  max  P(D) 

■{D} 

(c)  min  E  {  t(D)  } 

{D} 

SvT.  P(D)  >  0 

Q<D)  <  r 

(d)  max  P(D) 

{D} 

S.T.  E(t(D)  >  A 
Q(D)  <  r  . 


Formulation  (b)  may  be  solved  using  dynamic  programmings  the 
other  formulations  can  be  transformed  into  linear  programming 
problems  by  utilizing  the  techniques  described  in  Derman  (1962). 
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Pollock  (1964)  develops  search  strategies  to  minimize 
the  expected  cost  of  search  which  are  sequential  in  the  sense 
that  a  decision  at  any  time  is  dependent  upon  what  has  been 
observed  up  to  that  time.  The  search  process  is  represented 
in  terms  of  a  stochastic  dynamic  program  including  consideration 
of  false  alarm  probabilities.  The  optimal  search  strategies 
as  well  as  the  associated  minimum  costs  are  given.  The  state 
variable,  the  probability  that  the  target  is  present,  is  ad¬ 
justed  by  Bayes'  rule  after  every  observation.  It  is  shown 
that  the  optimal  sequential  strategy  is  similar  to  the  Wald 
sequential  probability  ratio  test.  The  target  is  assumed 
stationary,  although  if  the  target  is  not  yet  present  in  the 
region  of  interest,  it  has  probability  X  of  arriving  in  each 
successive  time  interval. 

Kadane  (1968)  studies  the  problem  of  choosing  a  strategy 
to  maximize  the  probability  of  finding  a  stationary  object 
when  a  budget  ceiling  is  imposed.  It  is  assumed  that  the 
probability  of  overlooking  the  object  in  the  search  of 

box  K,  given  that  it  is  in  box  K  and  has  not  been  found  before 
the  j^^  search  of  box  K,  is  a  function,  a^j^,  of  j  and  k  alone. 
Therefore,  the  (unconditional)  probability  that  the  j"**^  search 
of  box  K  is  conducted  and  is  successful  is  0  if  the  strategy 
does  not  include  a  j  search  of  box  K,  and  is 

TT  =  Pjk> 

j  '<  j 
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where  is  the  probability  that  the  object  is  hidden  in 
box  K.  Let  E. .  be  the  event  that  the  search  of  box  K 

J 

is  conducted  and  is  successful  and 


jk 


probability  that  th6  event  .dccurs. 


Let  a  denote  a  search  strategy,  then  the  probability  of 
finding  the  object  using  o  is  is  to  be  included 

in  the  summation  if  there  is  a  j^*^  search  of  the  box  in  o. 
The  simplification  and  extension  achieved  in  this  paper  are 
a  consequence  of  the  possibility  of  restricting  the  discussion 
to  the  unconditional  probabilities  of  these  mutually  exclusive 
events  It  is  possible  to  compute  the  conditional  proba¬ 

bility  that  the  search  of  box  K  will  be  conducted  and  will 
be  successful,  as, 


(r,s)E  6 


■rs 


where  6  is  the  set  of  searches  conducted  up  to  this  point. 

Let  the  j^^  search  of  box  K  cost  c.,  .  Then  the  largest 
cost  one  can  occur  using  strategy  o  is 


i:  c. 


jk 


where  c^j^  is  included  in  the  summation  if  there  is  a  j  search 
of  box  K  in  0.  In  short,  a  strategy  is  sought  to 


L 

o 


MAX 


*1-0 


Liie  adiiie  Toj.'  ectuh  oplxmetl 


X  and  is  a  single  point  or  a  closed  interval.  The  author 
describes  an  integer  programming  algorithm  (branch  and  bound 
variety)  adapted  to  the  problem  of  finding  the  object  subject 
to  a  budget  ceiling  C  when  diaoreteness  is  insisted  upon.^’ 

The  implications  of  the  previous  theory  towards  the  problem 
of  minimizing  the  expected  cost  are  summarized  in  the  following 
results. 

The  author  defines, a  set  of  searches  to  be  locally 
optimal  if  the  inclusion  of  (j',k')  and  exclusion  of  (j,k) 
implies 

"Vu-''.'  ■■  ■,'T  '  ■■  '  ... 

V-'  )  '  ^'k'  -  >  "hk 

"I'k'  “jk 

•  .1  .  • 

The  following  theorem  is  given: 

Theorem:  Let  Pjj^/Cjj^  be  non-increasing  in  j  for  each  k.  Any, 
locally  optimal  feasible  strategy  including  all 
searches  for  which  Pjj^  ^  0  minimizes  the  expected 
cost  of  all  unsucoessful  searches  plus  half  the  cost 
of  the  last,  successful  search.  .Such  a  strategy 


1 

For  Cj^  =  l,  k=l,2,‘'',N,  Chew  (1967)  gives  basically  the 
following  optimal  strategy:  To  maximize  the  probability 
of  finding  the  target  in  a  fixed  number,  N,  of  searches, 
choose  those  N  searches  (j,k)  for  .which  Pj^ 
largest. 


(l-cxj^)  is 
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exists  if  and  only  if 

Ca;  In  aiJ.  Doxes  x  for  which  ^  0  for  all  j 


b,  =  b  =  lim 
3-^“ 


where  b  >  0. 


(b)  If  b  is  positive  and  ^  some  (j,k), 

then  for  every  sufficiently  large  -j  ,  fjjy^j)^ 

b  or  0  in  each  box. 

(c)  _>  b  for  all  (j,k)  such  that  ^  0. 


Black  (1965)  presents  a  graphical  argument  for  the  optimal 
sequential  search  procedure  for  the  following  problem:  A 
stationary  target  is  in  one  of  n  regions.  It  is  in  region  i 
with  prior  probability  P^^,  a  look  in  region  i  costs  c^,  and 
the  target  can  be  overlooked  with  probability 


probability  that  the  taiget  is  found  on 
or  before  k^^  look, 

total  cost  of  the  first  k  looks,  and  c 
the  random  total  cost.  Then  the  expectation 
of  c  is 

00 

E(c)  =2  (c(k;-.c(k-l))  (l-P(k-l))  (7) 
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which  is  obtained  from 


Black  then  plots  P(k)  versus  C(k)  as  in  Figure  2. 


0  C(l)  C(2)  C(3) 


Figure  2 

Probability  of  Detection  in  K  Looks 


As  shown  in  equation  (7 )the  expected  cost  of  a  search  using 
this  policy  is  equal  to  the  shaded  area.  It  is  noted  that 
all  policies  with  finite  expected  cost  have  the  same  triangles 
in  their  probability-cost  plot,  with  only  their  order  changed. 
The  heights  of  the  triangles  are  given  by 

Pjd-aj)  ./-I 

and  the  base  by  c^.  Clearly,  the  policy  that  places  the 
triangles  in  order  of  decreasing  steepness  is  optimal,  if 
it  is  feasible. 
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Consider  aii  the  numbers 


“i 

c. 

1 


n-1 


arranged  in  a  two-dimensional  array.  Note  that  the 


P^(l-a^)o^ 


n-1 


"i 


are  monotone  decreasing  in  n. 

It  is  observed  that  the  application  of  Bayes*  rule  shows 
that  the  policy  with  minimum  expected  cost  is  identical  with 
that  generated  by  the  rule: 

"Always  look  in  the  region  for  which  the  posterior 
probability  (given  the  failure  of  earlier  looks)  of 
finding  the  object  divided  by  the  cost  is  maximum." 

Since  the  logarithm  is  monotone  increasing  in  its  argument, 
one  can  construct  the  optimal  policy  by  arranging  the  numbers: 


log 


'i 


+  (n-1)  log 


in  decreasing  order.  Viewing  these  numbers  as  points  along 
a  line,  the  points  corresponding  to  any  particular  region 
will  be  equally  spaced.  If  log  are  commensurate,  the 
optimal  policy  is  eventually  periodic. 


I 


r 
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Renyi  (1965)  considers  the  following  search  problem:  let 
Sj^  be  a  finite  set  having  n  ^  2  distinguishable  elements. 

Suppose  one  wishes  to  find  an  unknown  point  X  of  the  set  Sj^. 

It  is  further  assumed  that  one  cannot  observe  X  directly; 
however,  one  may  choose  some  functions  f  ^ ,  fj,  *•’,  fj^  from 
a  given  set  F  of  functions  defined  on  Sj^,  and  observe  the 
values  f^(x),  fjCx),  •••,  fj^(x).  It  is  assumed  that  F  contains 
M  functions,  M  ^  n.  A  strategy  of  search  is  a  method  for  the 
successive  choice  of  f^,  fj*  fj^,  which  leads  to  the  deter¬ 

mination  of  X.  The  usual  definitions  of  pure  and  mixed  strategies 
are  applied  to  the  choice  of  the  function  f^,  f^,  ••*,  fj^.  The 
author  attains  some  general  theorems  concerning  the  duration  of 
a  search  using  random  search  methods,  and  it  is  shown  that,  in 
general,  these  random  search  methods  are  almost  as  good  as  the 
best  pure  strategy,  and  are  usually  much  simpler. 

Miehle  (1954)  discusses  numerical  techniques  for  determining 
the  optimal  distribution  of  effort  under  constraints.  In  par¬ 
ticular,  one  has  various  types  of  effort  to  expend  on  corres¬ 
ponding  tasks  applied  towards  a  desired  result.  The  effect  is 
represented  by  E(x^,  Xj,  •••,  Xj^).  In  particular,  Miehle  studies 
the  case  in  which  the  effects  are  additive,  i.e., 

E(x^,  Xj,  •••,  Xj^)  =  +  f2(x2)  •••+ 

The  objective  is,  of  course, 

Max  E(x^  ,  Sj  ,  '  •  •  ,  Xj^) 

E-i  i 

Xi  ^  0. 


S.T. 


I 


w 


r 
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array,  the  columns  of  which  represent  the  efforts  '  ’  *  > 
and  the  rows  the  allowable  allocation  to  each  effort  ty^)e 
ranging  from  0  to  C,  for  the  maximum  value  of  E(x^,  Xj,  •••> 
Staroverov  (1963)  considers  the  following  search  problem. 

A  point  is  located  in  the  cell  with  probability  Pj^,  ^  ~ 

One  cell  is  inspected  per  unit  of  time;  if  the  point  lies  in  the 
cell  being  inspected,  it  is  discovered  with  probability  q  >  0. 
The  results  of  such  investigations  are  considered  independent. 
Let  denote  the  number  of  the  cell  being  investigated  at  time 
t,  if  the  point  was  not  discovered  up  to  the  time  t-1.  Let 
a  =  (Cj^,  ••',  denote  the  search  strategy  and  the 

time  required  for  discovering  the  point.  In  this  paper,  a 
procedure  of  searching,  o'*'',  is  determined  so  that 


E(T^rt)  =  inf  E(T^). 

0 

Arkin  (1964a)  extends  the  results  of  Staroverov  and 
considers  simultaneous  search  of  a  number  of  cells.  Explicit 
formulae  are  given  for  the  optimal  strategy  of  search  and  for 
the  corresponding  distribution  and  mean  value  of  its  duration. 

In  another  paper,  Arkin  (1964b)  considers  the  problem  of 
obtaining  uniformly  optimal  strategies  in  the  context  of  the 
stationary  search  problem.  The  a  priori  distribution  of  a 
particle  in TR^  is  given  by  the  density  function  f(x).  The 
search  strategy  is  defined  by  the  function 
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a 


a(x,t),  cr  >  Oi 


a(x,t)dx 


1. 


Let  Pp(T)  denote  the  probability  of  finding  the  particle  using 
strategy  a  during  time  T.  A  strategy  a*  is  uniformly  optimal 
if 


Pp*<T)  =  sup  P^(T)  for  any  T  >  0. 

a 


In  a  vei^y  general  case,  the  author  proves  the  existence  of 
the  strategy  a*  and  is  able  to  find  its  explicit  form. 

Chew  (1967)  considers  the  following  variation  on  the 
stationary  search  problem.  Let  the  a  priori  distribution 
of  the  object's  location  be  denoted  by  k  =  1,  2,  n, 

where 


k 


p 

k 


1-  ci  <  1. 


\ 

Since  in  this  case  the  search  has  a  positive  probability  of 
never  terminating,  one  must  couple  a  stoppinj^  rule  S  with  any 
search  procedure  o.  A  loss  function  is  defined  by  imposing  a 
penalty  cost  (c  >  0)  on  the  searcher  for  stopping  before  the 
object  is  found.  A  procedure  (  o,.  S)  which  minimizes  the 
expected  cost  to  the  searcher  (i.e.  which  yieldT  Bayes'  risk) 
is  derived. 
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MacQueen  and  Miller  (1960)  deal  with  the  problem  of 
wliether  or  not  a  search  activity  should  be  started  and,  if 
started,  whether  or  not  it  should  be  continued.  Their  model 
gives  rise  to  a  general  functional  equation  foi’  which  existence 
and  uniqueness  conditions  are  given. 

Gluss  (1961)  considers  a  model  in  whicli  there  are  N  neigh¬ 
boring  cells  in  one  of  which  there  is  an  object  that  it  is 
required  to  find.  The  a  priori  probabilities  of  the  object 
being  in  cells  1,  *•*  ,  N  are  respectively,  and 

the  costs  of  examination  of  these  cells  are  c^ . .  The 

search  policy  is  considered  to  be  optimal  when  the  statistical 
expectation  of  the  total  cost  of  search  is  minimized.  It  is 
assumed  that  costs  comprise  a  travel  cost  dependent  upon  the 
distance  from  the  last  cell  examined,  in  addition  to  a  fixed 
examination  cost.  It  is  assumed  initially  that  the  searcher 
is  next  to  cell  1,  =  i  +  c,  where  c  is  constant,  and  from 

then  onwards  (assuming  that  the  j"^^  cell  has  just  been  examined) 

=  |i"j|  c.  An  optimal  search  strategy  is  found  in  the  case 
where  the  Pjj^'s  are  all  equal,  and  an  approximately  optimal  search 
strategy  is  found  in  the  case  where  P^  is  proportional  to  i. 

The  latter  case  has  application  to  defense  situations  where 
complete  searches  occur  at  successive  intervals  of  time>  and 
hence  the  enemy  objects  are  thinned  out  the  nearer  they  come 


to  the  defense  base. 
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Pcllcck  (19G3)  cOiiBxuei'a  a  target  moving  in  a  Markovian 
fashion  between  two  regions.  The  objective  funtions  for  the 
standard  problems  of  the  minimization  of  the  expected  time 
until  detection  and  maximization  of  the  probability  of  de¬ 
tection  under  a  constraint  on  search  effort  are  derived.  For 
certain  special  forms  of  the  transition  matrix,  decision  rules 
are  derived  for  the  minimum  expected  time  problem.  Upper 
and  lower  bounds  are  also  derived  for  the  minimum  expected 
time  problem. 
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Neuts  (1963)  develops,  among  other  things,  stationary 
minimax  strategies  for  a  multistage  search  game.  A  stationary 
strategy  for  the  hunter  is  an  n«tuple 


y  =  (y^i  **  *•  y^); 


Vi  i  0.  2  Vi  = 

3=1 

which  denotes  a  probability  distribution,  chosen  once  and 

for  all,  and  by  which  the  region  to  be  examined  at  each  stage 

is  selected.  A  mixed  strategy  for  the  stationary  target  is 

an  n-tuple  x^Cx^^,  •  *  •  ,  x^) ,  with  _>  0;  (i  =  1,  2,  •  ’  •  ,n) 
n 

and^E^  Xj  s  l,  denotes  the  probability  of  the  target  being 

in  the  i^h  box.  If  the  searcher  uses  the  stationary  strategy 

y  and  the  target  the  mixed  strategy  x,  then  the  expected  return 

to  the  target  at  each  stage  of  the  game  is  given  by 

n 

A(x,y)  =  2  » 

k=l 

where , 


c^  =  cost  to  the  searcher  for  a  look  in  region  i 

=  probability  of  finding  the  target  given  the 
correct  region  is  searched 

a  =  reward  to  searcher  for  detecting  the  target. 
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-1 

i  =  1,  2,  •••, 

. 

Note  the  independence  of  y°  (searcher's  strategy)  of  all 
parameters  except  the  detection  probabilities 

The  same  remark  holds  for  the  expected  duration  of  the 
game,  i . e . , 

k  [  l-p(x°.y°)  1 


It  should  be  noted  that  stationary  minimax  etratejries  correspond 
to  the  following  oases: 

(a)  a  memoryless  s-earcher  .  . 

(b)  the  target  is  allowed  to  move  after 
each  region  is  searched. 

Let  X  =  (x^,‘  x^)  denote  an  arbitrary  mixed  strategy  for 

the  target.  Suppose  one  is  interested  in  determining  the  optimum' 
sequential  response  for  the  searcher  against  x  and  for  the  mini¬ 
mum  expected  loss.  Bellman's  principal  of  optimality  implies 
that  the  following  functional  equation  must  be  satisfied 

f^(x)  =  min  ic.  -  a  q^x.  +  5 (1-q^x. )f 

1<  i  <  n'  ^ 
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with 

c.>0»  0<'q  <l,  0<i5<3,  a>0  and 

Tj.x  a  ;  =>  •  •  • ,  tj,) 

defined  by 


The  n"tupl«?  C  ii?  the  a  posteriori  distributioti  derived  from  x, 
given  that  one  uneuecesitfui  search  of  box  i  .was  made.  For 
U.  <  C  <  !,' Bellman  (19S7)  settles  the  quei^tiohs  of  existence, 
uniqueness,  and  icontlnuity  of  the  solution  of  the  above  func- 
tiobal  eqtiation.  Neuts  obtains  the  following  rtesults  on  this 
equalrion  for  6  =  1.  Let  be  defined  by 

fj^+l(x)  »  min 

1_<  i  ^  n 

Theorem; 

The  sequence  f^^Cx),  n  «  0,  1,  ***,  for  6  i  j,s  monotone 
decreasing  in  n  for  all  x  e  X.  A  sufficient  condition 
for  this  to  be  true  for  all  x  and  Ci  <  <  1  is  that 


[ci  aq.x^  tfiCl-q.x.]  f^(x)j 


k=l  "Ik  " 


Theorem: 


There  exists  a  bounded  concave  solution  f(x)  to  the 
functional  equation  for  6  »1. 


Charnes  and  Schroder  (1967)  develop  models  and  methods 
to  find  optimal  tactics  in  an  idealization  of  antisubmarine 
warfare,  viewed  as  a  game  of  pursuit  between  the  hunter-killer 
force  and  a  possible  submarine.  The  status  of  the  pursuit  at 
every  move  t(t  -  1,  2,  •••)  is  taken  to  be  one  of  a  finite 
number  of  possible  states.  A  state  summarizes  the  tactical 
information  available  to  both  players  for  decision  making. 

A  finite  collection  of  tactical  plans  (decisions)  is  associated 
with  each  state.  When  the  players  move  they  each  choose  a  plan 
and  thereby  jointly  determine  an  intermediate  payoff  from  the 
hunted  to  the  hunter  and  a  transition  probability  distribution 
over  the  states.  The  objective  is  to  find  an  optimal  strategy 
for  each  player.  A  strategy  is  a  decision  (possibly  randomized) 
for  each  state  and  move,  an  optimal  strategy  is  one  of  a  minimax 
pair  for  the  total  expected  payoff.  These  concepts  are  presented 
in  terms  of  a  terminating  s:ochastic  game  (TSG)  which  may  be  de¬ 


fined  as  a  game  played  in  a  sequence  of  moves.  At  each  move,  the 


game  is  said  to  be  in  one  of  a  finite  number  of  states 


i  =  1,  n.  Tf  the  game  is  in  state  i  ( i  “  1 ,  2 ,  • • • ,  n ) 

and  the  hunter  chooses  alternative  K,  while  the  hunted  chooses 

KT 

alternative  L,  then  the  payoff  from  hunted  to  hunter  is 
(K  =  1,  2,  • • • ,  M. ;  L  =  1,  2,  • • • ,  N^). 

The  choice  of  alternatives  K  and  L  also  determines  the 
transition  probabilities: 


Under  the  above  assumptions,  the  game  terminates  with  probability 
1  and  the  accumulated  payoffs  received  by  either  player  are 
bounded.  A  behavior  strategy  for  either  player  is  An  n- tuple 
of  probability  distributions  x  =  (x^,  x^)  where  x^  = 


If  the  hunter  uses  a  behavior  strategy,  he  chooses  the 
mixed  strategy  x^  whenever  the  game  is  in  state  i  regardless  of 
what  move  it  is  or  the  manner  of  arrival  at  state  i.  By  choosing 


a  starting  statu  i  wp  oLtain  an  infinite  (th'=  number  of  moves 
may  not  Le  bounded)  game  G^(i  =  1,  2,  •••,  n).  A  terminating 
s  'mubaa  Lie  Kdine  is  defined  as  a  collection  G  =  (G-j^i 
Let  w.  denote  the  value  of  G.;  the  minimax  of  its  total  expect'd 
payoffs.  The  value  of  G  may  be  defined  to  be  the  vector 

A  A  A 

w  =  (w^',  Consider  a  two-person  zero-sum  game  with 

payoff  matrix  A^(oi)  where  A^(a),  i  =  1,  ‘  ,  n  is  the  x 

matrix  whose  K-L^  element  is 


n 


j  =  l 


and  a  =  (p'^,  •  •  •  ja^^)  is  an  n-vector  of  real  numbers. 

Let  VAL(B)  denote  the  minimax  value  of  the  two-person 
zero-sum  game  with  payoff  matrix  P  and  let  X(B)  and  YCB)  denote 
the  sets  of  optimal  mixed  strategies  for  the  respective  players. 
The  following  theorems  characterize  the  optimal  solutions  to 
the  terminating  stochastic  game. 

Theorem  1; 

The  value  of  the  terminating  stochastic  game  G  is  the 

A 

unique  solution  w  of  the  nonlinear  system  of  equations 
w^  =  VAL  [A^(w)]  i  =  1,  2,  •••,  n. 

Theorem  2; 

The  behavior  strategies  x,  y  where  x  =  x^[A^(w)], 
y  e  y^CA^Cw)]  (i=l,2,‘‘‘,  n)  are  optimal  for  the  first 
and  second  players,  respectively,  in  every  game  belong 


to  G. 


Charnes  and  Schroder  then  show  that  the  non] inear 
problem  can  oe  replaced  by  a  sequence  r*  linear  programming 
problems,  Stopping  criteria  are  developed  which  insure  the 

A 

desired  approximation  to  w.  The  preceding  results  are  then 
applied  to  a  problem  in  antisubmarine  warfare.  It  is  shown 
that  the  objective  function  of  the  minimization  of  the  expected 
duration  of  the  search  can  be  expressed  in  terms  of  a  terminating 
stochastic  game.  In  the  event  the  hunter  knows  or  is  willing 
to  assume  certain  behavior  on  the  part  of  the  submarine,  the 
game  becomes  a  one-person  game.  In  this  case,  the  determination 
of  the  hunters  optimal  strategy  is  reduced  to  solving  a  dis¬ 
counted  Markovian  decision  process  of  the  type  studied  by 
Howai'd  (1960).  Finally,  the  authors  study  a  finite  terminating 
stochastic  game  which  terminates  in  n  moves  or  a  terminal  state, 
whichever  occurs  first.  It  is  shown  that  in  this  case  the 
optimal  strategies  depend  upon  the  move  and  are  not  behavior 
strategies. 

Norris  (1962)  considers  the  two-sided  extension  of  a 
one-sided  search  problem.  The  search  is  conducted  against 
a  conscious  evader  who  is  able  to  observe  the  searcher's 
actions  and  capitalize  on  any  errors  he  makes.  The  evasion 
device  of  moving  between  looks  is  treated.  The  game  is  zero- 
sum  and  incorporates  a  fairly  general  reward  structure  which 
can  include  discounting.  The  reward  coefficients  associated 
with  this  structure,  as  well  as  the  location  of  the  boxes  and 


their  detection  orobabi 1 i ti o? ^  kncwn  Lu  both  players. 

Good  strategies  are  developed  for  the  players  when  the  gair.c 
involves  two  boxes.  In  the  case  of  an  infinite  moving  cost, 
designated  by  G*,  exact  solutions  may  be  obtained  when  the 
escape  probabilities,  and  Oj*  (the  nonipiements  of  the 
detection  probabilities)  satisfy  the  reDationship 


for  a  pair  of  intergers  n^  and  n^.  This  relationship  is  the 
necessary  condition  for  an  ultimately  periodic  optimal  program 
derived  by  Matula  (1964). 

In  the  case  of  a  finite  moving  cost,  designated  by  G, 
the  evaders  position  (2  Box  Case)  as  the  search  progresses 
is  described  by  a  probability  vector.  If  the  probability 
that  he  is  in  one  box  becomes  sufficiently  high,  he  should 
move  from  this  box  with  a  certain  probability.  This  causes 
the  probability  vector  describing  his  position  to  be  trans¬ 
formed  to  the  nearest  boundary  of  the  no-move  region.  The 
searcher's  good  strategy  can  be  generated  by  a  finite  Markov 
process.  In  some  states  of  the  process  the  next  look  is  made 
deterministically.  In  others  called  mixed  states,  the  next 
look  is  made  according  to  a  probability  distribution.  As 
moving  costs  increase,  deterministic  looks  are  made  more 
frequently,  and  the  situations  in  which  a  move  is  admissable 
occur  less  frequently.  In  the  case  of  infinite  (prohibitive) 
moving  costs,  the  searcher  makes  a  random  selection  from  two 


infinite  aedrcn  sequences.  Once  this  choice  has  been  made, 
the  search  process  is  completely  deterministic. 

Tn  the  M*.box  formulation  cf  the  finite  moving  cost  game, 
the  good  search  strategy  cannot  be  generated  by  a  finite 
Markov  process.  A  limited  memory  approach  to  finding  an 
approximation  to  the  good  search  strategy  is  suggested  for 
future  research.  In  the  game  designated  G“ ,  no  such  cost  is 
incurred  by  the  evader  when  he  moves.  As  a  result,  the  searcher 
cannot  gain  any  inference  concerning  the  evader's  position 
from  his  past  sequence  of  unsuccessful  looks,  and  each  look 
should  be  made  according  to  the  same  probability  distribution 
(this  is  the  stationary  minimax  case  discuso^d  by  Nouts  (1963)). 
When  the  N-box  form  of  G®  was  considered,  it  is  noted  that  the 
good  search  strategy  may  be  useful  when  the  evader  arrives 
sometime  after  the  start  of  the  game  or  leaves.  Finally,  it 
is  noted  that  the  results  for  the  N-box  case  in  which  the 
position  of  evader  is  specified  by  a  probability  vector  known 
to  the  searcher  may  be  useful  in  studying  some  one-sided  search 
problems. 

Johnson  (1964)  considers  the  following  search  problem: 

Blue  chooses  a  region  i  (i  =1,  2,  •**,  n)  in  which  to  hide. 

Red  selects  one  of  n  regions  to  search;  if  unsuccessful,  he 
is  told  whether  he  is  too  high  or  too  low,  and  repeats  until 
he  determines  the  correct  region.  Detection  occurs  with 
probability  one,  given  the  selection  of  the  region  chosen  by  Blue. 


R] 


Although  such  a  scenario  ir  -.ilitaiy  uujiLexi, 

it  may  be  quite  the  opposite  in  an  information  retrieval 
context.  Theorems  concerning  the  necessary  conditions  for 
optimality  are  presented-  Optimal  strategies  are  obtained 
(trial  and  error)  for  n  _<  11.  For  larger  problems  one  has 
recourse  to  linear  programming  techniques  on  a  digital  computer. 
If  J  (j  =  1,  2,  • • • ,  n)  is  a  vector  containing  the  proba¬ 
bilities  with  which  Blue  selects  the  j region,  then  it  is 
shown  that  P,  P?‘  ~  (S..}  denote  the  i^^  strategy 

for  Red,  i.e.,  equals  the  number  of  look  when  region  j  is 

searched  under  strategy  i.  The  following  theorems  pertain  to 
Red's  optimal  strategies. 

Theorem; 

Assume  at  given  stage  that  Red,  playing  S^,  has  located 
Blue  within  the  region  k  _<  j  _<  M,  and  that  calls  for 
the, next  look  at  a,  left  of  Blue's  frequency  distribution 
on  this  liiLei'val,  and  if  a  is  too  small,  next  playing  at 
b  to  the  right  of  a.  Then  a  necessary  condition  for  the 
optimality  of  against  {Pj}'is  that 


Thr:o»^t;ni ; 


At  each  stage  Red  should  make  his  guess  inside  the 
middle  third  of  Blue's  probability  distribution 
on  the  current  interval  of  uncertainty. 

Giammo  (1963)  considers  the  following  problem:  Consider 
two  opposing  mobile  battle  forces  that  are  able  to  change 
position  only  at  fixed  time  intervals,  not  necessarily  equal. 
Each  force  knows  the  area  in  which  the  other  is  operating 
and  is  assumed  to  be  efficiently  searching  this  area  for  the 
enemy's  position.  Labeling  the  forces  Blue  and  Rod,  Giammo 
defines  R  to  be  the  total  area  of  Blue's  operating  region 
and  assumes  that  Red  can  search  a  region  of  area  r  dt  in  a 
time  interval  dt,  where  r  is  some  constant.  B  and  b  are 
defined  in  a  similar  fashion  with  reference  to  Red's  operating 
region  and  Blue's  rate  of  search.  It  is  assumed  that  the 
Blue  force  moves  periodically  every  jt]j^  time  units'-with '  the 
first  move  occurring  at  random  with  a  uniform  probability 
density  in  the  time  interval  0  <_  t  £  tj^^.  Each  move  is  con¬ 
sidered  to  be  instantaneous  and  to  terminate  with  equally 
likely  probability  at  any  point  in  its  own  operating  region. 

It  should  be  noted  that  each  time  Red  (Blue)  moves,  a  new 
stage  of  the  search  starts  which  is  independent  of  the  pre¬ 
ceding  stages. 


\ )  .i 


The  objective  of  this  paper  is  to  develop  expressions 
for  the  probability  that  Red  will  detect  Blue  without  Blue's 
Having  previously  detected  Kedj  and  the  probability 

that  Blue  will  detect  Fed  vrithcut  Red's  having  pi'cvloualy 
detected  Blue, 

Define : 

Pj^(t)  =  the  probability  that  Blue  has  discovered 
Red  before  time  t, 

P^(t)  =  the  probability  that  Red  has  discovered 

Blue  before  time  t. 


In  these  definitions,  it  is  assumed  that  the  searchers  are 
independent,  i.e.,  that  the  discovery  of  Red  by  Blue  does 
not  interfere  with  continuation  of  Red's  search  and  visa-versa. 


Given  that  and  Pj,<t)  represent  the  integrals  of  corre-- 

spending  probability  density  functions,  one  can  write. 


Pb(T) 


P]^(t)dt, 


P^(t) 


Certainly,  one  can  obtain  the  probability  that  Blue  will  dis¬ 
cover  Red  before  time  t  without  Red's  having  discovered  Blue  as 

T 

=  /*  [1  -  PpCt)]  Pj^(t)dt, 

*b 


1 


V. 


and  i or  Red 


T 

P  .  Ct)  =  /  [1  -  r.  (t)  J  p  ( (:)dt. 

r,b  J  b  ^r 

The  desired  parameters  are: 


and 


P  —  P  ^  * 

r,b  r,b  ’ 


P  —  P  ^  ®  ^ 

b,r  b,r  ^ 


Integration  by  parts  yields 


F 


I 


r.b  -  J  Pr^^>  Pb(t)dt, 
o 


and 


b,r 


00 

/ 


Pb(t)  p^(t)dt. 


Giammo  then  derives  exact  as  well  as  approximate  expressions 
for  b  Pjj  r'  above  assumptions  concerning  the 

motion  and  search  structure  of  the  problem. 

Koopman  (1963)  presents  some  of  his  original  work  (Koopman, 
1957)  in  terms  of  a  zero-sum  game.  He  considers  the  problem 
of  detecting  an  enemy  unit  located  at  a  point  x  in  some  region 
R  with  a  limited  amount  of  search  effort  One  is  interested 

in  determining  a  distribution  of  random  search  intensity  il'Cx), 
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with  the  provision  that 

J"  ^(x)dx  =  4',  (J)(x)  >  0. 

R 

According  to  the  law  of  random  search  (Koopman  (1957)),  the 
probability  of  detecting  the  target  at  x  is 

1  -  e 

and  therefore  the  probability  of  detecting  the  target  when 
its  probability  of  being  at  x  has  density  pCx)  is 

P  ^  f  dx. 

R 

If  the  searcher  assumes  that  his  distribution  of  effort  ((i(x) 
is  known  to  the  target,  and  the  target  can  then  choose  his 
position  (or  position  density  p(x))  to  minimize  the  probability 
of  detection  P,  then  the  searcher  can  select  4t(x)  to  achieve 
maximum  P.  Conversely,  the  target  may  not  know  (()(x)  and  may 
assume  that  the  searcher  knows  p(x)  and  selects  (J)(x)  to  maximize 
P,  In  both  cases,  for  (j)(x)  =  $/R,  one  has 


maximum  P 


minimax  P 


In  the  heterogeneous  case  in  which  the  "visibility",  g(x), 
uCpcnia  upon  position,  the  pi-u-hahili ly  of  detecxion  becomes 


P  =  /  P(x)  Cl  - 

R 

and  the  constraint  on  search  effort  is  weighted  by  position, 
i.  e . , 


4i(x)dx 


♦  ,  h(x)  (J)(x)  >  0. 


Koopman  obtains  the  following  result  for  this  case:  The  target's 
strategy  p(x)  is  given  by  p(x)  =  a  h(x)/g(x),  and  the  search 
density  function  (|i<x)  =  b/g(x).  The .  constants  a  and  b  in  the 
above  expressions  are  determined  from 

i  -  f  ^  h-h  ■ 

R 

.  .  -®/R 

The  value  of  the  game  is  still  1  -  e 

The  case  of  a  moving  target  is  also  considered.  The 
target  has  to  move  along  a  path  C  from  a  point  x'’  on  a  given 
curve  to  a  point  x^  on  the  given  curve  K^,  C  passing  ■*-hrough 
a  field  R  (bounded  by  the  given  curves)  in  which  the  search  is 
being  conducted.  The  searcher  can  choose  any  4i(x)  subject  to 


<t>(x)  dx  = 


(J)(x)  >  Oi 
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and  the  target  can  select  his  curve  C,  which  he  follows  at 
a  constant  speed.  It  is  shown  that  the  expression  for  the 
probability  of  detection  is  given  by 


P  (c,  <J))  =  1  -  exp 


ifr(x)g(x)ds 


where  ds  is  the  arc  length,  and  the  integration  denotes  a 
line  integral  along  the  path  C.  Since  P(c,  increases  or 
decreases  with  ^ <|)(x)g(x>ds ,  the  problem  of  minimax  can  be 
stated  in  terms  of  this  line  integral. 

Beltrami  (1961)  studies  a  random  patrol  on  a  straight 
line  and  gives  a  rigorous  mathematical  discussion  leading 
to  the  paradox  that  the  requirement  of  uniform  coverage  in 
a  random  patrol  where  the  searcher  has  fixed  speed  imposes 
the  condition  of  a  non-random  back  and  forth  patrol.  The 
following  scenario  is  considered:  A  search  craft  S  patrols 
a  linear  barrier  in  some  back  and  forth  manner.  Using  de¬ 
tection  gear  it  has  an  effective  search  radius  p  (definite 
range  law)  which  is  assumed  small  in  comparison  with  tne 
barrier  length.  The  penetrator  P,  approaches  to  within  some 
distance  of  the  barrier  and  appraises  the  patrol  pattern  of  S. 
If  the  patrol  is  regular,  then  an  intelligent  tactic  on  the 
part  of  P  is  to  coincide  its  barrier  crossing  with  the  moment 
in  which  S  will  be  moving  away  or  is  at  the  extreme  distance 
from  the  cross-over  point.  A  random  patrol  for  S  is  chosen  in 


68 


order  to  completely  eliminate  any  advantage  to  P;  it  being 
essential  that  the  probability  that  a  given  point  is  covered 
in  a  move  by  S  is  as  nearly  constant  as  possible.  It  is 
shown  that  this  policy  will  assure  that  the  maximum  penetra¬ 
tion  threat  of  P  is  minimized. 

Dresher  (1961)  considers  two  formulations  of  a  recon¬ 
naissance  problem.  In  the  first  model,  it  is  assumed  that  the 
attacker  and  defender  have  two  strategies  each.  Blue,  the 
attacker,  wishes  to  seize  a  defended  enemy  position.  It  is 
assumed  that  he  has  two  courses  of  action: 

(a)  Attack  with  the  entire  force, 

(b)  attack  with  part  of  his  force,  leaving  the  remainder 
as  reserves  and  a  rear  guard. 

Let  the  payoff  matrix  A  be  given  by 


11 

®12 

21 

^22 

where,  for  example,  aj-j^  represents  the  value  to  Blue  if  he 
attacks  with  part  of  his  force  and  Red  defends  with  his  entire 
force. 

It  is  further  assumed  that  the  attacker  can  send  out  a 
detachment  to  reconnoiter  in  an  attempt  to  discover  the  plans 
of  the  defender.  In  order  to  defend  himself  against  such 
possible  action,  the  defender  may  take  counter  measures.  The 
new  game  now  has  16  strategies  for  the  attacker  and  4  for  the 
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defender.  The  matrix  for  the  new  game  can  however,  by  testing 
for  dominance,  be  reduced  to  a  4  x  4  matrix.  A  particular 
reconnaissance  game  is  solved  by  way  of  illustration. 

Dresher's  second  example  deals  with  the  value  of  recon*- 
naissance  information  in  the  context  of  a  bombing  attack.  It 
is  assumed  that  there  is  an  uncertainty  concerning  the  worth 
of  a  target.  Such  uncertainty  may  arise  from  unknown  or 
partially  known  results  of  earlier  strikes  on  the  same  target. 

If  the  exact  worth  of  the  target  is  discovered  through  re¬ 
connaissance,  then  it  is  possible  to  dispatch  the  most  efficient 
size  attacking  force  against  it.  In  order  for  a  reconnaissance 
to  be  successful,  at  least  one  reconnaissance  aircraft  must  fly 
to  the  target  and  return.  The  following  notation  is  introduced: 

B  =  Military  worth  of  one  bomber. 

R  =  Military  worth  of  one  reconnaissance  aircraft. 

T  =  Military  worth  of  the  target. 

({)(t)  =  Probability  that  the  value  of  the  target  does  not 

exceed  t.  This  probability  distribution  is  known 
prior  to  reconnaissance. 

r  =  Number  of  reconnaissance  aircraft  sent  out  prior 
to  the  mission. 

b  =  Number  of  bombers  dispatched  to  the  target  during 
the  mission. 

p  =  One-way  survival  probability  of  bomber  and  recon¬ 
naissance  aircraft  between  base  and  target. 
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oT  =  Probable  worth  of  the  target  after  being  hit  by 
one  bomber. 

o‘T  =  Probable  worth  of  the  target  after  being  hit  by 
two  bombers. 
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For  these  optimal  values  the  payoff  is  given  by 

M(r*,b*)  =  -  D  -  Din  ^  i  [AP  -  (l-P^)R]  -  (l-p^)Rr*. 

Isaacs  (1965)  discusses  extensions  of  his  theory  of 
differential  games  to  games  with  incomplete  information, 
e.g.,  search  games.  It  is  shown  that  when  the  hidden  objects 
are  numerous  and  immobile,  the  time  to  find  them  (payoff 
function)  is  nearly  independent  of  the  searcher's  strategy 
as  long  as  no  effort  is  wasted  re-searching  territory  already 
scouted  and  the  overlook  probability  is  zero.  In  the  case 
of  search  games  with  mobile  hiders,  Isaacs  conjectures  that 
the  details  of  v'  j  randomization  are  unimportant,  but  certain 
basic  parameters,  such  as  the  hider's  speed,  are  not.  He 
argues  that  in  either  case  there  appear  to  be  strong  grounds 
for  an  approximate  theory. 


4.0  MISCELLANEOUS  TOPICS 


(1964)  cor.sidcrc  "tb.c  following  aux'veilleince 
problem.  A  region  of  the  ocean  is  to  be  kept  under  sur¬ 
veillance  to  determine  the  probable  number  of  enemy  sub¬ 
marines  in  the  region  and  their  locations.  It  is  desired 
to  estimate  additional  measures  of  effectiveness  of  the 
surveillance  operation,  such  as  the  expected  fraction  of 
submarines  in  the  region  being  tracked  at  a  given  time. 

He  IS  also  interested  in  determining  how  the  above  measures 
depend  upon  the  capabilities  of  the  various  components  of 
the  detection  and  tracking  forces.  The  following  assump¬ 
tions  are  iriade: 

(a)  Submarines  enter  the  region  at  a  known  rate.  It  is 
also  assumed  that  their  time  on  station  is  a  random 
variable  with  known  distribution. 

(b)  Two  modes  of  detection  are  considered: 

(1)  Detection  at  barrier  line,  the  detection  process 
described  by  a  single  probability  of  detection, 

(2)  Area  search  detection,  the  detection  capabilities 
are  described  by  two  search  rates,  one  applies  to 
submarines  not  previously  detected,  the  other  to 
previously  detected  submarines. 
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(n)  r.ortflot  hsi-wpAn  1-h^  i-rarVAri  anH  +yt*  s’jbiT.ar’ir.s  car. 
be  broken  and  reacquisition  occur,  both  events  are 
described  by  their  respective  rates.  It  is  also 
assumed  that  a  contact  is  passed  from  a  detection 
unit  to  track  unit  with  probability  one  in  zero  time 
units . 

In  order  to  characterize  the  surveillance  system,  Dobbie 
describes  the  following  state  space: 

1.  Submarine  is  being  tracked, 

2.  Submarine  not  being  tracked,  contact  has  been  lost, 

3.  Submarine  not  detected. 

Using  renewal-type  arguments ,  Dobbie  derives  expressions 
for 

(a)  The  expected  number  of  submarines  in  the  i'*’*‘  state 
at  time  t,  the,  expected  number  of  submarines  in 
the  region  at  t, 

(b)  The  probability  that  a  submarine  in  the  region  is 
in  state  i  at  time  t. 

The  author  then  relaxes  the  assumption  that  contacts  are 
passed  from  detection  units  to  tracking  units  in  zero  time 
with  probability  one.  In  addition,  the  following  assumptions 
are  also  made: 
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(a)  Given  detection  by  a  barrier  unit  or  by  an  area 
sensor,  the  detecting  unit  will  attempt  to  main¬ 
tain  contact  until  a  tracking  unit  arrives  in  the 
vicinity . 

(b)  Targets  can  be  reacquired  either  by  area  search 
or  by  special  search.  If  contact  is  regained  by 
special  search,  it  is  assumed  that  tracking  will 
be  accomplished  by  the  detecting  unit  until  trans¬ 
fer  is  made  to  a  similar  unit  and  during  this  time, 
the  rate  of  losing  contact  is  X. 

In  this  case  the  expanded  state  space  includes: 

(1)  Targets  (submarines)  tracked  by  a  mobile  unit  in 
the  vicinity  of  the  target j 

(2)  Target  previously  tracked,  contact  recently  lost, 
local  search  being  made  to  regain  tracking  contact  •, 

(3)  Target  previously  tracked,  new  detection  r«scently 
made  by  area  search,  tracking  units  enroute  to 
area  or  searching  in  an  effort  to  obtain  tracking 
contact^ 

(4)  Target  previously  tracked,  search  to  regain  contact 
discontinued,  no  new  detection; 

(5)  Target  not  previously  tracked,  recently  detected  by 
area  search,  tracking  units  enroute; 

(6)  Target  detected  by  the  barrier  as  it  enters  the 
region,  tracking  units  enroute  or  searching  to  ob¬ 
tain  tracking  contact. 

(7)  Target  not  previously  tracked  and  no  previous  de¬ 
tection. 


/  0 


As  before,  Dobbie  develops  expressions  for  the  probability 
that  a  submarine  is  in  state  i  at  time  t,  given  thet  i-i- 
was  in  state  6  or  7  at  time  t  =  0  and  stays  in  the  region 
during  (0,t)j  i  =  1,  2,  7. 

Koopman  (1946)  developed  the  fundamental  theory  of 
target  detection  for  two  limited  cases.  In  the  one  case, 
the  detection  equipment  is  assumed  to  sweep  or  scan  at 
regular  intervals,  with  the  "glimpses"  of  the  target  long 
enough  apart  so  that  the  probability  of  detection  on  one 
glimpse  is  independent  of  the  probability  on  the  preceding 
glimpses.  In  the  other  case  the  detector  is  assumed  to  be 
continuous  in  its  action,  and  it  is  assumed  that  there  is 
a  probability  ydt  of  detecting  the  target  in  any  interval 
of  time  dt.  Kimball  (1963)  observes  that  actual  equipment 
in  use  has  detection  properties  which  lie  between  these 
limits.  He  shows  that,  in  spite  of  this,  actual  detection 
equipment  can  be  considered  as  equivalent  to  a  certain 
continuously  operating  detector  whose  properties  are  deriv¬ 
able  from  those  of  the  actual  equipment.  In  addition,  he 
also  considers  the  problem  of  holding  the  target.  Assuming 
the  detection  process  to  be  a  one-step  Markov  process, 
Kimball  notes  that  it  can  be  described  by  the  matrix 
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where,  e.g.,  gj^^^  is  the  probability  that  there  is  no  de¬ 
tection  on  a  given  scan  if  there  was  no  detection  on  the 
previous  scan.  New  parameters  r  and  g  are  defined  as 

^  "  ®oi  ®10  * 

_ gpl 

®  ■  8qi  +  SlO  ’ 

where  g  is  the  unconditional  probability  of  detection  on 
an  arbitrary  trial  and  r  is  a  measure  of  the  lack  of  cor¬ 
relation  between  trials.  It  is  shown  that  if  the  scanning 
frequency  is  f,  the  frequency  of  transitions  in  either 
direction  (from  the  detected  to  the  undetected . or; visa  versa) 
is 

w  =  frgCl  -  g)  . 

Kimball  defines  two  detectors  as  equivalent  if  their  g  and 
w  parameters  are  the  same.  In  particular,  any  detector  is 
in  this  sense  equivalent  to  a  continuous  detector  with  the 
following  properties .  If  the  detector  is  in  the  "undetecting" 
state,  the  probability  that  it  begins  to  detect  in  any  in¬ 
terval,  dt,  is  ydt,  and  if  the  detector  is  in  the  "detecting" 
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state,  it  has  a  probability  of  becoming  "undetected"  equal 
to  gdt.  The  proper  values  of  0  and  y  are 


Kimball  forms  the  following  model  of  tracking:  The  entire 
system,  detector  plus  operator,  can  be  in  any  one  of  four 
states : 

(1)  Detector  off,  target  not  tracked; 

(2)  Detector  on,  target  not  tracked; 

(3)  Detector  on,  target  tracked; 

(.4)  Detector  off,  target  tracked. 

it  is  assumed  that  the  behavior  of  the  detector  and  the 
operator  can  be  modeled  in  a  continous  fashion.  Let 

Adt  =  probability  of  a  transition  in  dt  from  state 
2  to  state  3,  and 

Mdt  =  probability  of  a  transition  in  dt  from  state  4 
to  state  1, 


The  state  diagram  is  given  by 
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Target  Not 
Tracked 


Target  Tracked 


The  steady  state  probabilities  of  being  in  the  four  states 
are  derived  in  terms  of  y,  8|  Mt  and  X,  as  well  as  the 
frequencies  with  which  both  the  tracking  and  detection  phases 
start  and  stop. 

The  problem  of  the  target  visibility  changing  over  time 
has  been  formulated  by  Bonder  (1969)  and  Disney  (1969). 

Bonder  considered  the  situation  in  which  the  target  and  the 
searcher  (detector)  may  not  be  continuously  visible  during 
the  period  of  time  in  which  the  searcher  is  examining  the 
subregion  containing  the  target.  The  searcher  has  a  de¬ 
tection  capability  only  when  the  target  is  visible.  The 
author  considered  the  following  situations: 

(a)  The  target  may  be  visible  to  the  searchers  for 
the  entire  search  interval  with  some  known 
probability  p, 

(b)  The  target  may  be  visible  at  the  start  of  the 
search  period,  the  length  of  the  visible  period 
being  a  random  variable  with  known  probability 
density  function,  and  not  reappear, 


(c)  A  single  period  of  visibility  may  be  exhibited 
starting  at  some  random  time  during  the  search 
interval  and  lasting  a  random  amount  of  time. 

In  each  of  these  cases,  the  probability  density  func¬ 
tions  for  the  time  until  the  first  detection,  the  time  spent 
searching  the  area  until  a  fixed  number  of  detections  occur, 
and  the  time  spent  searching  the  total  area  are  derived. 

Disney  characterized  the  visibility  process  in  which 
the  target  alternates  between  visible  and  invisible  states 
as  an  alternating  renewal  process.  The  transition  matrix 
for  this  process  is 

(  “  ). 

\  f2(t)  0  / 

where  f^^Ct)  is  the  probability  density  function  for  the 
time  in  the  visible  state  and  f2(t)  the  probability  density 
function  for  the  time  in  the  invisible  state. 

Employing  some  renewal  theory  arguments,  the  author 
obtained,  among  other  things, 

(a)  TT^Ct),  the  density  function  for  the  probability 
that  the  target  is  visible  at  time  t, 

(b)  for  a  fixed  time  interval  of  length  x^,  the  dis¬ 
tribution  of 

(1)  the  number  of  times  the  target  is  visible, 

(2)  the  total  time  of  visibility. 
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Analysis  of  interactions  between  the  visibility  and 
detection  processes  represents  an  important  extenMlon  of 
the  scope  of  knowledge  in  search  theory  related  to  the 
results  concerning  the  stationary  target,  Physically, 
the  structure  of  their  interactions  can  be  considered  as 
a  model  in  which  the  search  environment  acts  to  aid  the 
target,  e.g.,  the  terrain,  folllage,  etc.,  common  to  the 
subregion  in  which  the  target  is  operating,  or,  in  the 
ASW  context,  the  existence  of  termal  barriers,  and  other 
local  phenomena  which  tend  to  increase  (and  decrease)  the 
level  of  concealment  of  the  target  ov^  timeV "  In  the 
situation  in  which  a  single  interval  of  visibility  exists, 
the  probability  distribution  of  the  length  of  the  visible 
period  may  be  interpreted  as  the  time  required  for  the 
hunted  to  become  aware  of  the  hunter’s  presence,  Multiple 
periods  of  visibility  may  reflect  the  situation  in  which 
the  enemy  periodically  activates  some  form  of  sensing  equip¬ 
ment  which  makes  him  vulnerable  to  detection  by  the  searcher, 
Danskin  (1962a)  makes  a  study  of  the  optimum  distri¬ 
bution  of  aerial  reconnaissance  effort  against  land  targets 
in  the  presence  of  decoys.  The  model  considered  is  one  in 
which  the  reconnoitering  forces'  allocate  effort ‘among 
various  regions,  their  objective,  being  the  location  of  the 
targets,  assuming  the  side  being  reconnoitered  is  passive. 


The  information  function  of  communication  theory  is  chosen 
as  the  measure  of  effectiveness.  That  is,  the  information 
of  a  reconnaissance  is  defined  to  be  the  change  in  the 
uncertainty  of  the  region  resulting  from  that  reconnaissance. 
For  each  of  the  (K^)  regions,  one  has  an  information  function 
where  x  is  the  level  of  reconnaissance.  The  alloca¬ 
tion  problem  is  stated  as:  Given  X  units  of  reconnaissance 
effort  to  distribute  among  the  regions,  how  shall  this  be 
done  so  as  to  maximize  the  information? 

One  wishes  to  maximize 

subject  to 

■  ■ 

2xj^  =  X,  Xj^  i  0  . 

Ksl 

The  solution  to  the  problem  depends  entirely  on  the  form 
of  the  functions  Ij^(x).  Under  the  most  realistic  assumptions 
concerning  the  detection  probabilities  associated  with  aerial 
reconnaissance,  the  author  is  unable  to  determine  the  behavior 
of  the  second  derivative  of  and  thus  the  form  of  the 

objective  function.  In  Part  II  of  the  two-part  paper, 

Danskin  (1962b)  considers  the  two-sided  reconnaissance 
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problein»  in  which  the  side  being  reconnoitered  seeks  to 
minimize  the  ‘informeticn  (.rr^xisiizs  the  uunfusion)  obtained 
by  the  reconnoiterer ,  whi&e  maintaining  at  least  a  certain 
minimum  acceptable  threat  with  a  fixed  budget.  This  problem 
formulated  as  a  zero-sum,  two-person  game,  is  solved  for 
a  special  case  (fixed  equipment)  and  it  is  shown  that  there 
exists  a  solution  in  mixed  strategies  for  the  general  use. 

Smallwood  (1965)  considers  a  model  for  the  placement 
of  n  detection  stations  for  optimum  coverage  of  an  arbitrary 
area.  The  stations  are  assumed  to  be  identical  and  to 
have  a  probability  of  detection  that  is  a  function  only  of 
the  distance  between  the  station  and  the  event  to  be  detected. 
Furthermore,  stations  are  assumed  to  operate,  independently 
of  each  other.  It  is  also  assumed  that  the  enemy  has 
complete  knowledge  of  the  station  locations  and > effectiveness 
and  is  interested  only  in  eluding  detection  by  the  detection 
stations.  The  situation,  is  reduced  to  the  minimax  problem 
of  placing  the  stations  so  that  the  maximum  probability  of 
not  detecting  an  enemy  event  is  minimized.  Necessary  con¬ 
ditions  for  the  optimal  locations  are  given,  and  a  hill 
climbing  interative  technique  based  on  these  conditions  is 
described  in  some  detail.  The  technique  is  applied  to  the 
problem  of  the  location  of  detection  stations  within  the 
United  States  and  the  Soviet  Union. 
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Pollock  (1969)  points  out  that  there  has  been  a 
tendency  to  iuOuel  t'ue  Lux<ee  phases  of  a  general  surveillance 
operation  (search,  detection,  and  ensuing  aotion)  separately, 
the  output  parameters  of  one  such  model  are  often  used  as 
the  inputs  to  another.  He  considers  some  of  the  interfaces 
between  these  phases  and  presents  some  examples  of  the 
relation  between  search,  detection,  and  decision  theories 
involving  false  alarms,  continuous  surveillance,  localization, 
and  the  selection  of  appropriate  measures  of  effectivenessr 

W.  Edwards  (1962)  notes  that  the  development  of  a 
dynamic  decision  theory  will  be  central  to  the  expansion  of 
research  on  human  decision  problems.  A  taxonomy  of  decision 
problims  is  presented,  most  require  a. dynamic  theory  in 
which  the  decision-maker  is  assumed  to  make  a  sequence  of 
decisions,  ba.sing  decision  n  +  1  on  what  he  learned  from 
decision  n  and  its  consequences.  The  relevance  of  the 
mathematical  developments  in  dynamic  programming  and  Bayesian 
statistics  to  dynamic  decision  theory  is  examined. 

Along  these  lines ,  Rapoport  (1966)  considers  a  dynamic 
programming  model  of  a  controller,  i.e.,  a  dynamic  decision¬ 
maker,  who  can  actively  manipulate  the  environment  by  his 
decisions.  An  experiment  is  described  in  which  subjects 
were  given  dynamic  decision-making  tasks,  the  results  fit 


well  the  analytic  solution  obtained  from  the  dynamic 
programming  model. 

Smallwood  (1966)  notea  that  in  many  practical  situ¬ 
ations  the  discount  factor  for  future  rewards  and  costs 
is  not  known  precisely.  The  dependence  of  the  optimum 
policy  on  the  discount  factor  is  often  noted  in  the  model¬ 
ing  of  these  problems.  He  discusses  the  dependence  of  the 
optimum  policy  on  the  discount  factor  for  the  class  of 
finite-estate,  time-invariant,  Markov  models.  A  procedure 
is  developed  for  finding  the  value  of  the  discount  factor 
for  which  the  decision-maker  is  indifferent  between  two 
policies.  The  procedure  is  extended  co  a  discussion  of  how 
one  can  find  the  complete  description  of  the  optimum  policy 
regions  over  any  range  of  the  discount  factor, 
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5.0  AREAS  FOR  FUTURE  RESEARCH 

Table  1  is  an  attempt  at  summarizing  the  current 
"b  Ldte  of  the  ar-L”  in  search  and  x'‘econnai8Bance  theory 
(at  least  in  subjects  relevant  to  the  goals  of  this  report). 
The  numbered  entries  refer  to  the  papers  in  the  bibliography 
given  in  Chapter  6.0.  No  attempt  was  made  to  enumerate 
all  the  papers  in  a  given  category >  but  only  to  indicate 
that  the  area  had  been  treated  in  the  literature, 

Examination  of  Table  1  clearly  reveals  areas  in 
which  little  or  no  research  activity  has  been  devoted  and 
which  are  considered  important ‘ topics  for  futui^e  research. 
These  are  briefly  noted  below  along  with  some  areas  suggested 
by  Pollock  (1969)  and  Dobbie  (1963). 

1.  Interaation  Betwean  Detection  and  Viaihility  Brocaaeaa 

The  table  suggests  that  the  visibility  problem  as 
defined  by  Bonder  (1969)  and  Disney  (1969)  has  not  been 
treated.  As  noted  in  the  text,  the  visibility  process  has 
been  modeled  as  follows: 

(a)  The  target  may  be  visible  to  the  searcher  for 
the  entire  search  interval  with  some  known 
probability  p. 

(b)  The  target  may  be  visible  at  the  start  of  the 
search  period,  the  length  of  the  visible  period 
being  a  random  variable  with  known  probability 
density  function,  and  not  reappear. 
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(c)  A  single  period  of  visibility  may  be  exhibited 
starting  at  some  random  time  during  the  nearnh 
interval  and  lasting  a  random  amount  of  time. 

(d)  The  target  may  exhibit  alternating  periods  of 
visibility  and  invisibility,  the  durations  of  each 
being  random  variables . 

These  forms  may  interact  with  all  modes  of  detection  and 
targets,  thus  giving  rise  to  many  research  possibilities. 
This  area  is  currently  being  studied  extensively  in  thfe  SRL 
under  this  ONR  contract . 


2.  Non-Stationary  Tavgeta 

(a)  Target  motion  independent  of  position  and  known 
u  precisely.' 

(b)  Target  motion  independent  of,  position  and  drawn 
at  random  from  a  population  known  to  the  searcher. 

(c)  Target  motion  dependent  upon  position  and  known 

precisely  to  the  searcher.  '> 

<d)  Target  motion  drawn  randomly  from  a  known  popula¬ 
tion  which  is  a  function  of  target  position. 

Oe)  Target  motion  chosen  in  advance  by  the  evader  from 
a  probability  distribution  known  to  the  searcher. 

(f)  Evader  chooses  motions,  subject  to  limitations 
known  partially  to  the  searcher,  throughout  the 
search  as  he  obtains  information  on  the  past  activ 
ities  and  location  of  searcher. 


3.  StruotuJ^»  and  Capabilitiee  of  Operational  Deteotora 

Altuuu^li  Liie  uebCiT'lptloii  uT  caab uiiip llunH  x'egax'uxng 

detectoDB  given  In  Section  li2  differentiated  between 
detectors  that  had  single- laiidmmdttiple-scan  capabilities, 
this  difference  is  not  reflected  in  the  diagram.  wWith  the 
exception  of  Kimball's  paper  (1963),  all  research! ipapara 
considered  in  this  literature  review  take  as  given  the 
capabilities  of  the  detector  and  do  not  distinguish  single¬ 
scan  versus  multiple-scan  effects.  Research  is  needed  in 
this  area  to  understand  the  behavior  of  operationally  use¬ 
ful  devices,  e.g.,  the  effect  of  multiple  scans,  independence 
between  successive  looks,  etc. 

4.  Optimization  criteria  will,  in  general,  depend  upon 
the  objective  of  the  operation.  If  additional  action  is 
to  be  taken  after  detection,  then  neither  the  maximization 
of  the  probability  of  detection  nor  the  minimization  of  the 
expec'ted  search  time  may  be  optimal  for  the  combined  opera¬ 
tion.  Research  should  be  devoted  to  the  structuring  of 

the  total  activity,  which  includes  search,  detection,  track, 
and  ensuing  action,  before  selecting  the  optimization  cri¬ 
teria.  For  example,  search  activity  can  readily  be  inter¬ 
faced  with  the  combat  activity  which  results  from  mutual 
detection. 


5.  The  output  of  many  of  the  optimization  seirch  studies 
has  been  the  fiaat/l 

.  -  ww  ocai  ca  in  a  oox,  one 

might  instead  consider  the  likely  possibility  that  the 
actual  search  time  will  be  a  random  variable  and  examine 
its  effect  on  the  optimal  policy.  It  is  not  unlikely  that 
searchers  may  have  various  modes  of  operation,  each  of 


which  has  a  characteristic  distribution  of  search  time  as 
well  as  associated  Type  I  and  II  errors. 


6.  The  likelihood  that  searchers  will  not  (or  cannot) 
rollow  optimal  search  procedures  suggests  research  be  devoted 
to  the  problem  of  converting  theoretical  results  into 
practical  rules  of  application. 
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